This application is based on patent application No. 11-283670 (1999) filed Oct. 5, 1999 in Japan, the content of which is incorporated hereinto by reference.
1. Field of the Invention
The present invention relates to an input device of 3-D translation and rotation and its method used for inputting 3-D translation and rotation to a data processing system such as a computer or to an amusement system.
2. Description of the Related Art
Recently, with an increase in processing speed of computers and development in computer graphics (CG) technique, 3-D input technique has been finding wide application in various fields rapidly. In particular, demand for 3-D input devices sharply increases in applications in the Internet typified by consumer entertainment systems or VRML (Virtual Reality Modeling Language). In computer graphic design or CAD systems, a mouse or tablet is very popular as an input device. In addition, to carry out manipulation, observation and modification of object models from an arbitrary direction, demand for inexpensive devices for viscerally inputting 3-D translation and rotation is escalating.
Today, there are various types of 3-D input devices such as those based on measurement of magnetic field, photometric stereo and acoustic time-of-flight, and a joystick. The devices based on the measurement of magnetic field and photometric stereo carry out desired measurement of translation and rotation using a transmitter and receiver. The joystick includes a precision mechanical link to convert its movement to 3-D position information. The photometric stereo obtains 3-D position information from information of parallax using multiple TV cameras.
In addition, a device is proposed which employs four photo detectors and a display unit to obtain translation and rotation of the photo detectors (see, Japanese Patent Application Laid-open No. 9-325007 (1997), for example). The device utilizes the calculation method disclosed in 20 Japanese Patent Application Laid-open No. 7-270124 (1995), enabling 3-D measurement with an inexpensive lens and a small number of photo devices. Since this method carries out 3-D measurement optically, it can achieve stable 3-D input without any effect of magnetic field or temperature.
Since the conventionally available 3-D input device based on the measurement of magnetic field measures variations in spatial magnetic field, it is vulnerable to metallic parts, electronic devices and geomagnetism. On the other hand, since the acoustic time-of-flight device measures the phase or time-of-flight of sonic waves, it is vulnerable to atmospheric pressure or temperature. In addition, both of them require a transmitter and receiver, thereby increasing the device size. Furthermore, since the joystick must carry out translation and rotation independently, it is inappropriate as a visceral 3-D input device.
Finally, since the 3-D input device employing the four photo detectors and display unit (Japanese Patent Application Laid-open No. 9-325007 (1997)), for example)has a minimum number of photo detectors necessary, even omission of a single detection point on the display disables the 3-D measurement. Furthermore, since the calculation method disclosed in Japanese Patent Application Laid-open No. 7-270124 (1995) markedly reduces its sensitivity when the display surface directly faces the photo detectors, or when a line connecting parallel two of the four photo detectors is parallel to the display surface at the photo receiving side, the device is limited in its application. Moreover, since the 3-D measurement cannot be carried out until the detection of all the four photo detectors has been completed, the 3-D measurement interval is limited to within the interval between the scanning time of the display unit.
An object of the present invention is to provide an input device of 3-D translation and rotation and its method that can carry out stable visceral 3-D input over a wide range at arbitrary measuring time intervals without the effect of electric field or temperature with requiring small installation space.
To accomplish the foregoing object, according to a 1st aspect of the present invention, there is provided an input device of 3-D translation and rotation for measuring 3-D translation and rotation of a photo device with respect to a display emitting a bright spot, the input device of 3-D translation and rotation comprising: a display with a display surface on which a bright spot is movable: a photo device including a condense unit for condensing a bright spot on the display to be projected onto a projection plane on which multiple photo detectors are disposed for detecting a projection position of the spot light on the projection plane; correspondence means for bringing a bright spot display position on the display into correspondence with a projection position of the spot light on the photo detectors; and an extrinsic parameter calculator for calculating the 3-D translation and rotation of the photo device with respect to the bright spot moving surface of the display from a set of multiple bright spot display positions and their projection positions which are brought into correspondence by the correspondence means.
According to a 2nd aspect of the present invention, the photo detectors for detecting the projection position of the spot light may consist of a set of photo detectors, each of which includes a spot-like photo window.
According to a 3rd aspect of the present invention, the photo detectors with spot-like photo windows may consist of three photo detectors, and the bright spot display positions and the projection positions, which are brought into correspondence by the correspondence means, may have three positions, each.
According to a 4th aspect of the present invention, the photo detectors with spot-like photo windows may consist of at least five photo detectors, and the bright spot display positions and the projection positions, which are brought into correspondence by the correspondence means, may have at least five positions, each.
According to a 5th aspect of the present invention, the correspondence means may comprise: means for bringing a projection image position on the photo detectors into correspondence with projection time, the projection image position being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; means for recording the projection position and projection time; and a correspondence detector for bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
According to a 6th aspect of the present invention, the display may carry out scanning and display of the bright spot.
According to a 7th aspect of the present invention, the display may employ a CRT display for carrying out scanning and display of the bright spot.
According to an 8th aspect of the present invention, the display may employ a ray projection device and projection board for carrying out scanning and display of the bright spot.
According to a 9th aspect of the present invention, the photo device may comprise multiple sets of condense units and projection planes, the multiple sets having different optical axes.
According to a 10th aspect of the present invention, the photo device may comprise means for mounting it on a mobile unit such as a robot and vehicle, or for wearing it on part of a body, or for holding it by part of the body.
According to an 11th aspect of the present invention, the photo device may be combined with a device for converting 2-D movement into digital 2-D information.
According to a 12th aspect of the present invention, the device for converting may consist of one of a trackball, a mouse and a laser range finder.
According to a 13th aspect of the present invention, the photo device may be made freely movable by utilizing one of wired and wireless transmission of its output signal.
According to a 14th aspect of the present invention, the photo detectors for detecting the projection position of the spot light may consist of a photo detector including a flat photo detecting face for detecting the position of the spot light.
According to a 15th aspect of the present invention, the correspondence means may comprises: means for bringing a projection image position of the spot light on the photo detectors into correspondence with projection time, the projection image position of the spot light being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; means for recording the projection position and projection time; and a correspondence detector for bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
According to a 16th aspect of the present invention, the photo device may consist of a position sensitive light detector.
According to a 17th aspect of the present invention, the display may carry out scanning and display of the bright spot.
According to an 18th aspect of the present invention, the display may employ a CRT display for carrying out scanning and display of the bright spot.
According to a 19th aspect of the present invention, the display may employ a ray projection device and projection board for carrying out scanning and display of the bright spot.
According to a 20th aspect of the present invention, the photo device may comprise multiple sets of condense units and projection planes, the multiple sets having different optical axes.
According to a 21st aspect of the present invention, the photo device may comprise means for mounting it on a mobile unit such as a robot and vehicle, or for wearing it on part of a body, or for holding it by part of the body.
According to a 22nd aspect of the present invention, the photo device may be combined with a device for converting 2-D movement into digital 2-D information.
According to a 23rd aspect of the present invention, the device for converting may consist of one of a trackball, a mouse and a laser range finder.
According to a 24th aspect of the present invention, the photo device is made freely movable by utilizing one of wired and wireless transmission of its output signal.
According to a 25th aspect of the present invention, there is provided an input method of 3-D translation and rotation for measuring 3-D translation and rotation of a photo device with respect to a display emitting a bright spot, the input method of 3-D translation and rotation comprising the steps of: moving the bright spot freely on a display surface of the display: condensing the bright spot on the display, and detecting a projection position of the spot light projected onto a projection plane; bringing a bright spot display position on the display into correspondence with the projection position of the spot light on photo detectors; and calculating the 3-D translation and rotation of the photo device with respect to the bright spot moving surface of the display from a set of multiple bright spot display positions and their projection positions which are brought into correspondence.
According to a 26th aspect of the present invention, the photo detectors for detecting the projection position of the spot light may consist of a set of photo detectors, each of which includes a spot-like photo window.
According to a 27th aspect of the present invention, the step of bringing may comprise the sub-steps of: bringing a projection image position on the photo detectors into correspondence with projection time, the projection image position being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; recording the projection position and projection time; and bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
According to a 28th aspect of the present invention, the photo detectors for detecting the projection position of the spot light may consist of a photo detector including a flat photo detecting face for detecting the position of the spot light.
According to a 29th aspect of the present invention, the step of bringing may comprise the sub-steps of: bringing a projection image position of the spot light on the photo detectors into correspondence with projection time, the projection image position of the spot light being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; recording the projection position and projection time; and bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
According to a 30th aspect of the present invention, there is provided a computer readable recording medium for causing a computer to execute an input method of 3-D translation and rotation for measuring 3-D translation and rotation of a photo device with respect to a display emitting a bright spot, the computer readable recording medium causes the computer to execute the steps of: moving the bright spot freely on a display surface of the display; condensing the bright spot on the display, and detecting a projection position of the spot light projected onto a projection plane; bringing a bright spot display position on the display into correspondence with the projection position of the spot light on photo detectors; and calculating the 3-D translation and rotation of the photo device with respect to the bright spot moving surface of the display from a set of multiple bright spot display positions and their projection positions which are brought into correspondence.
According to a 31st aspect of the present invention, the photo detectors for detecting the projection position of the spot light may consist of a set of photo detectors, each of which includes a spot-like photo window, and the step of bringing may comprise the sub-steps of: bringing a projection image position on the photo detectors into correspondence with projection time, the projection image position being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; recording the projection position and projection time; and bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
According to a 32nd aspect of the present invention, the spot light on the photo detectors at the step of bringing may be the spot light detected and output from the photo detectors, and the step of bringing may comprise the sub-steps of: bringing a projection image position of the spot light on the photo detectors into correspondence with projection time, the projection image position of the spot light being a position formed by projecting the bright spot on the display onto the projection plane of the photo device; recording the projection position and projection time; and bringing bright spot display positions at multiple times into correspondence with their projection positions in accordance with the recorded projection time.
The input device of the 3-D translation and rotation in accordance with the present invention can consist of only the semiconductor device capable of detecting the projection image position, and the display for moving or scanning the bright spot on its 2-D display surface so that the input device can be implemented which requires only small installation space and can achieve reliable 3-D measurement free from the influence of the magnetic field or temperature. By measuring the 3-D translation and rotation at three points using the three photo detectors, the 3-D translation and rotation can be measured at minimum time intervals. In addition, by bringing the bright spot display positions at any multiple times into correspondence with their projection positions on the photo device in accordance with the recorded time, and by calculating the 3-D translation and rotation of the photo device with respect to the bright spot moving surface or the bright spot scanning surface of the display from the set of the arbitrary number of the bright spot display positions and their projection positions, the calculation of the 3-D translation and rotation can be implemented from the arbitrary number of the bright spots at arbitrary positions and their projection image positions. This offers an advantage of being able to achieve the visceral 3-D input over a wide range without being affected by the scanning interval of the display.
The input device of the 3-D translation and rotation in accordance with the present invention can consist of only the semiconductor device capable of detecting the projection image position, and the display for moving or scanning the bright spot on a 2-D surface so that the input device can be implemented which requires only small installation space and can achieve reliable 3-D measurement free from the influence of the magnetic field or temperature. By measuring the 3-D translation and rotation at three points using the three photo detectors, the 3-D translation and rotation can be measured at minimum time intervals. In addition, by bringing the bright spot display positions at any multiple times into correspondence with their projection positions in accordance with the recorded time, and by calculating the 3-D translation and rotation of the photo device with respect to the bright spot moving surface or the bright spot scanning surface of the display from the set of the multiple bright spot display positions and their projection positions, the calculation of the 3-D translation and rotation can be implemented from the multiple sets of minimum 3 projection images abstracted from arbitrary number of the projection images at arbitrary positions and the associated bright spot display positions. This offers an advantage of being able to achieve the visceral 3-D input over a wide range without being affected by the scanning interval of the display.